Paper Review

August 12, 2026

[论文精读] OasisKV: Scaling In-Decode KV Cache

A memory-centric LLM inference system that decouples full KV-cache storage from HBM, using lookahead tokens from speculative decoding to prefetch only the most relevant KV blocks, achieving 1.69×-2.3× throughput gains within 0.7 points of full-attention accuracy.

KV Cache Sparsity LLM Inference Sparse Attention Speculative Decoding Prefetching Disaggregated Serving Memory Management